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ABSTRACT 

The Center for the Study of Evaluation (CSE) model 
identifies conditions under which evaluators intervene in educational 
programs, provides for a sequence of valuation activities and 
conditions under which departtires from this sequence occurs, and 
defines the relationship between the evaluator and the decision 
maker. This paper provides a study of these aspects of the CSE model 
in the context of the evaluation of a preschool serving American 
Indian children in south-central Los Angeles, and draws conclusions 
about the efficacy of the model based on the degree to which it 
eliminates extraneous detail and highlights key elements of the 
educational process. (Author/DB) 
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In addition to specifying five stages of. evaluation, the model devel- 
oped by the Center for the Study of Evaluation (CSE) makes a number of 
theoretical assunptions about the evaluation process. This paper investi- 
gates three of these assumptions in relation to the. evaluation of a pre- 
school designed primarily for urban American Indian children in south-central 
Los Angeles. 

Description of the School 

Sixty of the 90 three to five year old children, enrolled in the Tribal 
American Preschool are Indian (the remaining 30 are roughly evenly divided 
among C3iicanos and Whites; there are no Blacks or Orientals). Twenty tribes, 
each distinct from one another in language and custOTis, are represented, 
^proximately ten percent of the children do not or.wilj. not speak English. 
All are eligible for and receiving aid from the Department, of Public Social 
Services. Half the children attend school in the. morning; half attend in 
the afternoon. IXiring each session, two-thirds of. the, children are assigned 
to two teachers who are developing a team- teachings approach with the assis- 
tance of two Indian aides. The remaining children, are. assigned to an 
ijistructor v^o is develqping a Montessori program with the assistance of a 
Chicano aide. Other staff include a direct or,. a. woman. lAo acts as book- 
keeper, secretary and receptionist, a registrar,: and; two. cooks, all of whom 
are Indian, as well as a Chicano custodian and a part r time. nurse who is White. 

The fimding proposal submitted to the State.of .California. outlines a 
cOT^prehensive set of goals for the school. These include gross and fine 



2 



motor skills, reading and math readiness, development. of attitudes conducive 
to successful public school attendance, and. awareness, of. the. local community. 
In addition, a series of related services. are. outlined, including medical, 
dental and optometric examinations, a nutritional .program^, instruction in 
personal hygiene, and training of teacher aides -to. provide family social 
services and encourage family participation in, school /activities. The 
proposal outlines three different instructional-modes-to. achieve these goals, 
with a view toward identifying methods viiich will .enhance, the educational 
prospects of urban American Indian children. These. instructional modes are 
identified as team- teaching, Montessori, and, cooperative.^ Goals, services 
and modes of instruction were described with.varying.degrees-of. specificity 
and a promise that the curriculum would be adapted to Indian values and 
culture. 

The proposal identifies three major questions vrfiich must be answered 
by those developing the Preschool. First, has. the program been installed 
as specified in the proposal? Second, is each. mode. of. instruction operating 
as efficiently as possible? Third, what aspects.. of. each, mode of instruction 
should be used in developing the single mode \Mch-will maximize the oppor- 
tunities of diildren enrolled in the school. 

T he CSE Evaluation Model 

The third, fourth and fifth stages of the .CSE. evaluation, model provide 
a useful guide to selecting, collecting, analyzing and reporting information 

^Partial funding prevented development of the cooperative mode. 
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needed in making these decisions. The basic features of each stage of 



the model are briefly described below: 



IMBEDS ASSESSMENT involves Stating the potential range of objectives or goals, 
deciding which of these are of highest priority, and determining how well 
these objectives are currently being met. The latter information is then 
used to identify the major needs. 

PROGRAM PLANNING liivolves making decisions about the kinds of programs 
that should be adopted to meet the problems identified in the. needs assess- 
ment. During the program planning phase, the evaluator. suggests. techniques 
to facilitate planning decisions, provides advice regarding. evaluation 
requirements for alternative plans, and builds into the final plan the 
procedures necessary for conducting subsequent evaluation. activities^ 

IMPLEMENTATION EVALUATION focuses on whether the procedures specified in 
the program plan are actaally carried out in the intended manner.. Thus, it 
involves assessing the degree to which the program plan has been ad£ff)ted 
properly to the field situation. 



PROGRESS EVALUATION IS aimed at determining the extent to which the program 
is actually making gains towards achieving its d)jectives^ Since ..a. program 
may be implemented exactly as planned but still not. reach. its .intended ob- 
jectives, it is necessary to investigate v^ether the program is a good one 
for meeting the needs. 

OUTCOME EVAWATiON leads to a final judgment regarding the general worth of 
the total program. (Klein, Fenstermacher, Alkin, 1971) • 



Conduct of the Evaluation to Date 

A variety of techniques were used to collect Implementation Evaluation 
data. Sixteen half-day observations divided evenly among afternoon and 
morning class sessions were conducted by the three-person evaluation team 
during a four-week period three months after the school opened. Interviews 
with teachers and the school director, and information collected from school 
records, completed the data collection phase. A report making recommenda- 
tions to help the staff complete iii?)lementation of the program was submitted 
to the director at the end of January, 1973. 
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Intervention in Educational Programs 

One characteristic which distinguishes evaluation from traditional re- 
search is the willingness of evaluators to intervene in programs which they 
are studying. CSE specifies that intervention is appropriate and. desirable 
during the first four stages of the model, and is normally in the. form of 
reports providing decision makers with information useful in inproving edu- 
cational programs. Intervention which reduces the quality. of . information 
obtained, or compromises the objectivity of the evaluator, is viewed as 
undesirable. 

Believing that it was necessary to allow the children. to. become, accus- 
tomed to us if we were to administer tests to them, we helped. them« with 
untied shoes, with moving heavy objects, with puzzles they. could. not solve, 
talcing advantage of every opportunity to talk to a child for a-mcment or two. 
Within two weeks we all knew the majority of the children, by name, and spent 
much of our time talking with them, playing with them, helping them. Occas- 
ionally one of us would step in to prevent serious injury, as lAen one child 
was trying a hammer on another *s head, or v^ien one was esc^ing toward a 
busy street. The teachers learned to make use of the Navajo. member. of the 
evaluation team to work with one or two non-English speaking Navajo, children. 
Each of us now has favorites among the children. We have conferred with 
teachers about difficulties they were having vdth individual children, on 
the basis of our own work with each child. IVingard and Kosecoff (1973) 
have noted the extent to which they became involved in prograin administration; 
in the present case, we have become involved in the program itself. Clearly, 
we have ceased to be inpartial observers. 
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Conpensation came through conparability of information collected be- 
cause we identify students by name in our logs, and through our ability to 
quiz children informally to determine what they are learning, from their activi- 
ties. In view of the reliability and validity of tests available, for pre- 
school children, the problem of consistency between published, tests and goals 
of the school and language and ailtural problems involved in testing at the 
Preschool, we believe, but cannot prove, that we have increased -both the 
quantity and qiality of infomation which we have about the school. We plan 
to make more systematic use of the technique to collect progress evaluation 
data during the second year of the program. 

Departure from Normal Sequence 

The basic CSE evaluation model consists of five stages /vAich.nonnally 
are conducted in sequence as the program develops. However, Alkin (1969) 
suggested that evaluation results from any one of these stages. might, lead to 
reconinendations for repeating program activities associated. with. earlier 
steps of the model. Further, Klein, et al. (1972), noting that in actual 
practice teachers are involwd in activities associated with several stages 
of the model at one time, suggested that evaluation may have. to be. based on 
several stages of the model simultaneously. These modifications have become 
part of the model. 

Receiving only partial funding, the Preschool was unable to inplement 
one of the three modes of instruction described in the proposal. The 
initial Inplementation Evaluation report indicated that physical facilities 
did not meet program requirements; that, while there was overlap in some 
areas, there were clear differences between the goals pursued by the tv;o 
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modes of instruction; that adaptation to Indian values and custans had 
noc begun; that enrollment was beLjw expectations and community involve- 
ment was nonexistent. [Lest these comnents lead you to conclude that 
the school is a failure, it should be emphasized that these problems 
have been largely corrected in the past month. Enrollment is up to ca- 
pacity; parents are volunteering as aides; facilities, materials, equip- 
ment and supervision are inproved; student activities are more purpose- 
ful. The school is now a functioning institution.] 

A major recommendation of the Inplementation Evaluation was for 
clarification, expansion and inprovement of the goals and objectives in 
the original proposal. Evaluation of this activity requires recycling 
to Needs Assessment, the first stage of the model, vdiile con^leting the 
Implementation Evaluation. It is anticipated that both Program Planning 
Evaluation and Progress Evaluation will be required next year. Thus, 
both recycling and sirmiltaneous conduct of two or more evaluation stages 
were found to be necessary in practice, as suggested by the model. How- 
ever, clarification as to the conditions under which recycling should 
occur, as well as guidelines for allocating evaluative resources when 
simultaneously conducting two or more stages of the model, are required. 

Relationship between Evaluator and Decision Maker . 

The model defines the evaluator's responsibility as selecting, col- 
lecting, analyzing, interpreting and reporting information needed by a 
decision maker. The decision maker is responsible for identifying the 
purposes for \rfiich he requires information, and for determining, on the 
basis of such factors as organizational resources and effect on other 
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commitments, whether the information can be put to use. It appears that 
the recommendations in the Implementaticm Evaluation report were accepted, 
and that the changes suggested have been or are being made, without assess- 
ment as anticipated by the model. One excepticm exists in recommendations 
made concerning organization of playground activities, made orally several 
days before inclusion in the Written report. These recommendations were 
j'uli/ discussed by the director, an Indian member of the school advisory 
board, and me member of the evaluation team. Apparently, altering the 
mediim by which evaluative information is reported will preserve the rela- 
tionship between evaluator and decision maker described in the model. It 
is tenq)ting to claim this as adapting the model to the oral traditions of 
Indian culture; certainly, it is an adaptation to the situation. If such 
adaptations are not made, the evaluator may find that he has become the 
administrator rather than the evalustor of the project, an experience re- 
ported by Wingard and Kosecof f (1973) . 

Relationship Between the CSE Model and an Educational Program. 

CSE assumes that the evaluation model .includes all possible types pf 
educational decisions, that an educational program begins with a rational 
selection of appropriate educational goals; that it is possible to select 
an appropriate means of achieving the goals selected; and that in^rovements 
can be made and the effects of programs estimated by a variety of measure- 
ment techniques. 

Each day at the Tribal American Preschool, teachers try to keep an eye 
on fifteen or more active an unpredictable children with ten minute atten- 
tion spans, remaining alert to prevent injury and breakage, arbitrating disputes. 
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distracting the aggressive, helping the puzzled, encx)uraging the slow, 
cleaning up spills, preparing materials for classroom activities, read- 
ing to children, leading games... When these actiAdties are eliminated, 
teachers appear to plan and review their day-to-day activities in a man- 
ner closely alcin to that described by the model. Ihe practicality of 
the model stems from its usefulness in isolating the learning plan of 
teachers from the distractions provided by the problems of sipervising 
a classroom of children. 

Summary 

The CSE model identifies conditions under vAiidi evaluators inter- 
vene in educational programs , provides for a sequence of evaluation act- 
ivities and conditions under vrtiich departures from this sequence opcurs, 
and defines the relationship between the evaluator and the decision 
maker. This p^er has studied these aspects of the CSE model in the 
context of the evaluation of a preschool serving American Indian child- 
ren in south-central Los Angeles , and draws conclusions about the ef- 
ficacy of the model based on the degree to vAiidi it eliminates extraneous 
detail and higjiligjits elements of the educational process. 
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